Absolute Continuity/Singularity and Relative Entropy Properties for Probability Measures Induced by Diffusions on Infinite Time Intervals

Optimal measures and Markov transition kernels ∗ Roman

We study optimal solutions to an abstract optimization problem for measures, which is a generalization of classical variational problems in information theory and statistical physics. In the classical problems, information and relative entropy are defined using the Kullback-Leibler divergence, and for this reason optimal measures belong to a oneparameter exponential family. Measures within such...

Optimal measures and Markov transition kernels

We study optimal solutions to an abstract optimization problem for measures, which is a generalization of classical variational problems in information theory and statistical physics. In the classical problems, information and relative entropy are defined using the Kullback-Leibler divergence, and for this reason optimal measures belong to a oneparameter exponential family. Measures within such...

Entropy of infinite systems and transformations

The Kolmogorov-Sinai entropy is a far reaching dynamical generalization of Shannon entropy of information systems. This entropy works perfectly for probability measure preserving (p.m.p.) transformations. However, it is not useful when there is no finite invariant measure. There are certain successful extensions of the notion of entropy to infinite measure spaces, or transformations with ...

Absolute Continuity and Singularity of Probability Measures in Functional Spaces

Divergence measures based on the Shannon entropy

A new class of information-theoretic divergence measures based on the Shannon entropy is introduced. Unlike the well-known Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, their close relationship with the variational distance and the probability of misclassification error are ...